Due to specific characteristics of screen contents, coding tools were developed for the Screen Coding Extension of HEVC. The tools demonstrate significant gains in coding efficiency. Among them, the palette mode represents block of pixels using indices to the palette (major colors), and encodes the palette and the indices by exploiting spatial redundancy. A typical palette mode coding method is composed of two parts: 1) coding methods for the palette, and 2) coding methods for the samples using the palette. The latter part is composed of palette index coding, run length coding, and escape pixel coding.
A very simple but efficient histogram-based algorithm is used to classify pixels. Specifically, the most significant L peak values in the histogram are selected as major colors, and the pixel values that are close to a major color will be quantized to the major color. Other pixels which do not belong to any major color sets are escape pixels, which will also be quantized before coding.
For lossless coding, the quantization processes are not used. For each pixel, a color index is assigned to indicate which color set it belongs to. If L major colors are used, the values of major colors 0 to (L−1) are signaled for the set of L major colors and the value of major color N is signaled for the escape pixel set. The palette is generally implemented as a color lookup table in which each color entry is associated with an index. A single palette is used to code both luma and chroma components. Each entry represents a specific RGB(YUV) color. For example, entry 1 with (R, G, B)=(0, 0, 0) represents the pure black color while entry 0 with (R, G, B)=(2, 10, 200) represents a blue-ish color. When the video format is 420, the chroma plane is up sampled to generate the color palette table.
The palette-based coding is performed on a CU basis. To code the current palette using the palette predictor list as a reference, a binary vector is used to indicate whether each entry in the palette predictor list is reused in the current palette. The reused entries are placed at the beginning of the current palette, maintaining their order in the palette predictor list. This is followed by new palette entries which are not in the palette predictor list. A palette initializer with predefined entries can be optionally used to initialize the palette predictor list, resulting in improved coding efficiency in such a scenario. A palette initializer can be signaled in the picture parameter set (PPS) or the sequence parameter set (SPS).
After classification, the pixels of a block can be converted into color indices according to the major color set selected. A predictive coding method is applied to the indices, where a pixel line can be predicted by three different modes, including horizontal mode (i.e., copy index mode), vertical mode (i.e., copy above mode) and normal mode (i.e., escape mode). Two index scan orders are used when coding the color indices, namely horizontal traverse scan and vertical traverse scan. An index rotation flag is signaled to indicate which scan order is used.
In copy index mode, starting from the first pixel, one or more consecutive indices are copied from the first pixel. The index of the first pixel is signaled. In copy above mode, one or more consecutive indices are copied from above pixel line. In escape mode, when an escape pixel is encountered (signaled by the largest index in major color set), its corresponding pixel value is coded right after the index. There may be more than 1 escape pixels and in different color values in the CU. For different escape pixel locations, the pixel values of escape pixels may be different.
For each copy index mode, an index value is signaled. The index signaling is grouped in the front to improve the CABAC throughput. The escape values are signaled in the back due to the same reason. The copy index mode and copy above mode are signaled in between the indices coding and escape coding.
Traditionally, a coding unit contains samples of both the luma and chroma components. It was proposed in JVET that samples of chroma components may have an independent or separate split tree structure as compared to luma components. Typically, such a separate coding tree structure starts from the CTU level. Therefore, it is possible that a chroma CU (a CU that contains only two chroma components) can be larger than its luma counterpart at the corresponding sample location.